Auto-generate TypeScript Typings from Documentation #38

jvilk · 2016-02-08T20:46:23Z

I am opening this PR to start a conversation. It should not be merged as-is until we determine how to integrate these typings into nodegit.

Summary

I have created a script that creates valid TypeScript typings for NodeGit from the documentation JSON. As NodeGit improves its documentation, the TypeScript typings will improve as well.

This is what the working typings look like.

Usage

Run generate/index.js in the usual manner. It will generate ts/nodegit.d.ts.

To create a new project that uses this file, do the following:

npm install -g typescript tsd
mkdir test_proj
cd test_proj
npm install nodegit
tsd install node
cp path/to/nodegit.github.com/ts/nodegit.d.ts node_modules/nodegit/lib/

Edit node_modules/nodegit/package.json to include a "typings": "lib/nodegit.d.ts" field. This tells the TypeScript compiler where to find typings for your package.

Create tsconfig.json in test_proj:

{
  "compilerOptions": {
    "target": "es3",
    "module": "commonjs",
    "noImplicitAny": true
  },
  "files": [
    "typings/node/node.d.ts",
    "test.ts"
  ]
}

Create the test.ts in test_proj:

import Git = require('nodegit');

var getMostRecentCommit = function(repository: Git.Repository) {
  return repository.getBranchCommit("master");
};

var getCommitMessage = function(commit: Git.Commit) {
  return commit.message();
};

Git.Repository.open("nodegit")
  .then(getMostRecentCommit)
  .then(getCommitMessage)
  .then(function(message) {
    console.log(message);
  });

Compile test.ts:

$ tsc

Then, you can fiddle with the code in an editor that supports TypeScript, such as Visual Studio Code or Atom with the atom-typescript plugin. Simply open up the test_proj folder, and the editor will use tsconfig.json to orient itself and provide autocomplete using the typings.

Limitations

Currently, there are a number of weird type annotations that I have to smooth over or type to the 'any' type (see the getType function). Most of them are untyped callback functions. As you improve NodeGit's documentation, we can remove some of the logic in that function.

Suggestions

It would be ideal if:

You included generated TypeScript declaration files in nodegit's NPM module, as illustrated in the example.
You added a test to your testsuite that verifies that the produced declaration file compiles properly. If it does not, it likely indicates a documentation bug!

If the script breaks, ping me. In the worst case, you can ship w/o decls for a release. In the best case, you'll fall in love with TypeScript and decide to maintain it yourselves! 😄

An alternative proposal is to put the types in the DefinitelyTyped repository, where they can be maintained separately. But I'd argue that including them into the NPM module directly would be mutually beneficial -- automatic checking of your docs, and typings for TypeScript users!

Thoughts?

Currently, the declarations are not correct due to the incompleteness of the documented types. I will need to fix up some documented types, and add hacks that remove certain annotations.

…ions. There are now only a few issues left to deal with.

jvilk · 2016-02-08T20:48:41Z

Oh, and I can remove my name from the generated typings. I just added most of the standard headers that you see in other manually curated TypeScript typings without thinking.

jvilk · 2016-02-08T21:08:36Z

Updated sample file to use tsconfig.json, which is needed for editor support.

jvilk · 2016-02-10T15:48:25Z

Updated with a link to the typings, in case someone finds this and wants to use them immediately!

jvilk · 2016-02-10T16:42:45Z

Fixed generation to support optional parameters, and updated linked typings with the fix.

jvilk · 2016-02-10T16:54:42Z

Aw, fiddlesticks. My last commit broken the typings because some functions have optional parameters followed by required parameters. TypeScript supports this via multiple function declarations for the same function, so I'll need to retool the script a little bit.

Here's the older, working typings with non-optional types. I'll fix the optional type bug in a few hours.

…arameter list by generating variants. Handles complexities like Git.Commit.create.

jvilk · 2016-02-10T21:56:37Z

Fixed optional parameters, so complex functions like Commit.create are specified properly using multiple declarations. Link to typings updated in PR description.

johnhaley81 · 2016-02-15T16:00:43Z

Hey @jvilk sorry it's taken so long to get back to you. So my initial feeling about this is really good. Although we don't use TypeScript in GitKraken it's used in a lot of projects out there and if anything could benefit from type safety, it would be NodeGit.

So the main thing that stands out to me is grinding down this list. This stuff should be fairly straight-forward to generate (if the docs are good) and should require minimal changes as NodeGit/libgit2 evolves.

I really like the test idea to confirm that docs are good and I think that would be a must for this. NodeGit's biggest weakness is the docs and I think that would help with the overall health.

Anybody have any thoughts on this overall? /cc @tbranyen @maxkorp @Mr-Wallet @srajko @dkoontz

jvilk · 2016-02-15T16:17:01Z

For testing, you'll need to modify nodegit's testing configuration such that it:

Clones this repository
Symlinks nodegit into the generate folder
- ...which may be a problem on Windows. On some configurations, junction attempts (Windows symlinks) are met with a permissions error.
- Alternatively, you could modify your doc generation script so you can configure nodegit's location to avoid the symlink.
Runs the nodegit.d.ts generator
Installs the TypeScript NPM module
Runs the TypeScript compiler on an empty ts file that references nodegit.d.ts

You'll also need to include node.d.ts, the Node TypeScript declarations, in your repository or fetch it at test time. I recommend the former option to reduce your tests' network dependencies.

For prepublish to NPM, you'll also need a script that does the above to generate nodegit.d.ts, and you'll need to add a typings field to the package.json.

tbranyen · 2016-03-25T00:54:55Z

I'm gonna look into this either this weekend or next week! Thanks for making the PR @jvilk

johnhaley81 · 2016-04-27T23:51:26Z

@tbranyen did you ever get a chance to look into this? I'd like to plop this in for v0.13.0.

tbranyen · 2016-04-28T00:50:25Z

generate/lib/write_ts_declarations.js

+      type = type.slice(0, angleIndex) + '<' + getType(type.slice(angleIndex + 1, type.indexOf('>'))) + ">";
+    }
+
+    switch (type) {


Ideally we should eventually move this to a dedicated JSON file like the descriptor.json.

tbranyen · 2016-04-28T00:51:23Z

Left one comment, didn't review the actual generator code, but will trust that @jvilk tested the output.

tbranyen · 2016-04-28T00:52:30Z

My primary concern here is that we're pushing out code that TypeScript users will rely on and it will need love. Do we have any takers for maintaining?

joshaber · 2016-04-28T19:16:29Z

My primary concern here is that we're pushing out code that TypeScript users will rely on and it will need love. Do we have any takers for maintaining?

It's relevant to our interests (/cc https://github.com/github/ohnogit).

jvilk · 2016-04-28T19:35:27Z

One limitation I've noticed: I use PromiseLike as the return value of methods that return promises to avoid dependencies on additional typings or TypeScript's ES6 output mode, which includes typings for native browser Promises. But PromiseLike lacks some of the methods on the promises your library uses. (Example: I think it does not specify catch.)

Due to other commitments, I do not have time to look into this or other issues until mid-May. But I wanted to raise it in case someone else jumps in to take a look at this or improve it.

johnhaley81 · 2016-04-28T20:33:39Z

I'm mostly interested in a system that automatically lifts the typing out from our docs. That way we could automate everything. There are still issues with some docs but most of them are correct. Also, it provides a large incentive for keeping the docs up-to-date.

This IMO helps with the maintainability of the whole system. Ain't nobody going to want to manually maintain TypeScript typings.

joshaber · 2016-04-28T20:56:20Z

Maybe this is nuts, but I almost wonder if we should go the other way: generate docs from a TypeScript declaration. The TypeScript declaration syntax is gonna be more well-defined than any documentation formatting.

We could use this script to seed the declaration, and then going forward treat it as The Truth.

johnhaley81 · 2016-04-28T21:07:06Z

We could do that. The thing that I did like about the docs is that it's basically jsdoc declarations in the JS code and the rest is lifted from the idefs.json. So really the source of truth is the thing that we generate the C++ code from (plus JS helper methods).

joshaber · 2016-04-28T21:18:13Z

Ah right. I forgot it didn't all come from JS.

pburgmer · 2016-10-29T18:48:50Z

+1 for this feature. would be create to have these type informations

maxkorp · 2016-11-01T13:25:21Z

A significant portion of our documentation is incorrect (still directly references the libgit2 docs). Not incorrect enough that one cant manually reference it, but for autogenerating code, I'm not sure. I haven't looked at this closely, but if it's building off of the actual types we build out, that's sane data (as sane as the rest of nodegit, anyways.

jvilk · 2016-11-01T15:14:03Z

This builds off of your JSDoc data, which is sane once I've filtered out some invalid types. I've successfully used the typings produced by this tool in code that interacts with NodeGit.

tbranyen · 2018-07-17T03:04:23Z

I feel like this should be moved to the nodegit repo. I'm using TypeScript more now and think this would be great to land. But the documentation site doesn't seem like the right place for it.

John Vilk added 4 commits February 8, 2016 14:27

Adding code to generate TypeScript declarations.

3826813

Currently, the declarations are not correct due to the incompleteness of the documented types. I will need to fix up some documented types, and add hacks that remove certain annotations.

[TS decls] Include [EXPERIMENTAL] in descriptions where appropriate.

c3daad4

[TS Decls] Fixing enums, preventing bad things from entering declarat…

acb7b6b

…ions. There are now only a few issues left to deal with.

Typings now compile in TypeScript!

d0bc904

Properly mark optional parameters to functions.

f687656

[TS Decl] Correctly handle optional parameters in the middle of the p…

5d0e1c2

…arameter list by generating variants. Handles complexities like Git.Commit.create.

jvilk mentioned this pull request Apr 9, 2016

How to declare a statement in d.ts that has both an instance method and an instance attribute and has same name microsoft/TypeScript-Handbook#233

Closed

tbranyen reviewed Apr 28, 2016
View reviewed changes

johnhaley81 force-pushed the master branch from 55ea72f to f5c0718 Compare February 6, 2017 20:57

johnhaley81 force-pushed the master branch from f5c0718 to 990337f Compare February 6, 2017 20:59

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Auto-generate TypeScript Typings from Documentation #38

Auto-generate TypeScript Typings from Documentation #38

jvilk commented Feb 8, 2016

jvilk commented Feb 8, 2016

jvilk commented Feb 8, 2016

jvilk commented Feb 10, 2016

jvilk commented Feb 10, 2016

jvilk commented Feb 10, 2016

jvilk commented Feb 10, 2016

johnhaley81 commented Feb 15, 2016

jvilk commented Feb 15, 2016

tbranyen commented Mar 25, 2016

johnhaley81 commented Apr 27, 2016

tbranyen Apr 28, 2016 •

edited

Loading

tbranyen commented Apr 28, 2016

tbranyen commented Apr 28, 2016 •

edited

Loading

joshaber commented Apr 28, 2016

jvilk commented Apr 28, 2016

johnhaley81 commented Apr 28, 2016

joshaber commented Apr 28, 2016

johnhaley81 commented Apr 28, 2016

joshaber commented Apr 28, 2016

pburgmer commented Oct 29, 2016

maxkorp commented Nov 1, 2016

jvilk commented Nov 1, 2016

tbranyen commented Jul 17, 2018

Auto-generate TypeScript Typings from Documentation #38

Are you sure you want to change the base?

Auto-generate TypeScript Typings from Documentation #38

Conversation

jvilk commented Feb 8, 2016

Summary

Usage

Limitations

Suggestions

jvilk commented Feb 8, 2016

jvilk commented Feb 8, 2016

jvilk commented Feb 10, 2016

jvilk commented Feb 10, 2016

jvilk commented Feb 10, 2016

jvilk commented Feb 10, 2016

johnhaley81 commented Feb 15, 2016

jvilk commented Feb 15, 2016

tbranyen commented Mar 25, 2016

johnhaley81 commented Apr 27, 2016

tbranyen Apr 28, 2016 • edited Loading

Choose a reason for hiding this comment

tbranyen commented Apr 28, 2016

tbranyen commented Apr 28, 2016 • edited Loading

joshaber commented Apr 28, 2016

jvilk commented Apr 28, 2016

johnhaley81 commented Apr 28, 2016

joshaber commented Apr 28, 2016

johnhaley81 commented Apr 28, 2016

joshaber commented Apr 28, 2016

pburgmer commented Oct 29, 2016

maxkorp commented Nov 1, 2016

jvilk commented Nov 1, 2016

tbranyen commented Jul 17, 2018

tbranyen Apr 28, 2016 •

edited

Loading

tbranyen commented Apr 28, 2016 •

edited

Loading